Navigational guidance via computer-assisted fluoroscopic imaging

ABSTRACT

Digital x-ray images taken before a surgical procedure by a fluoroscopic C-arm imager are displayed by a computer and overlaid with graphical representations of instruments being used in the operating room. The graphical representations are updated in real-time to correspond to movement of the instruments in the operating room. A number of different techniques are described that aid the physician in planning and carrying out the surgical procedure.

RELATED APPLICATIONS

[0001] This disclosure is related to U.S. patent application Ser. No.09/106,109, entitled “System and Methods for the Reduction andElimination of Image Artifacts in the Calibration of X-Ray Imagers,”filed on Jun. 29, 1998.

FIELD OF THE INVENTION

[0002] The present invention is directed generally to image guidedsurgery, and more particularly, to systems and methods for using one ormore fluoroscopic X-ray images to assist in instrument navigation duringsurgery.

DESCRIPTION OF THE RELATED ART

[0003] Modern diagnostic medicine has benefitted significantly fromradiology, which is the use of radiation, such as x-rays, to generateimages of internal body structures. In general, to create an x-rayimage, x-ray beams are passed through the body and absorbed, in varyingamounts, by tissues in the body. An x-ray image is created based on therelative differences in the transmitted x-ray intensities.

[0004] Techniques are known through which x-ray images are used tolocate the real-time position of surgical instruments in the patientanatomy represented by the x-ray image without requiring x-rays to becontinually taken. In one such system, as disclosed in U.S. Pat. No.5,772,594 to Barrick, light emitting diodes (LEDs) are placed on a C-armfluoroscope x-ray imager, on a drill, and on a reference bar positionedon the bone to be studied. A three-dimensional optical digitizer sensesthe position of the LEDs, and hence the position of the drill, the C-armfluoroscope, and the object bone. Based on this information, thereal-time position of the drill in anatomy represented by the x-rayimage is determined, and a corresponding representation of the drill inthe x-ray image is displayed. This allows the surgeon to continuallyobserve the progress of the surgery without necessitating additionalx-ray images.

[0005] Surgical navigational guidance, as discussed above, can provide atool for helping the physician perform surgery. It is an object of thepresent invention to provide several enhancements to traditionalsurgical navigational guidance techniques.

SUMMARY OF THE INVENTION

[0006] Objects and advantages of the invention will be set forth in partin the description which follows, and in part will be obvious from thedescription, or may be learned by practice of the invention. The objectsand advantages of the invention will be realized and attained by meansof the elements and combinations particularly pointed out in theappended claims.

[0007] One aspect of the present invention is directed to an x-rayimaging device comprising a plurality of elements. In particular, thex-ray imaging device includes an x-ray source for generating cycles ofx-ray radiation corresponding to an image acquisition cycle; an x-rayreceiving section positioned so that x-rays emanating from the x-raysource enter the x-ray receiving section, the x-ray receiving sectiongenerating an image representing intensities of the x-rays entering thex-ray receiving section. Additionally, a computer is coupled to thex-ray receiving section and radiation sensors are located in a path ofthe x-rays emitted from the x-ray source. The radiation sensors detectthe beginning and end of a radiation cycle and transmit the detectedbeginning and end of the radiation cycle to the computer.

[0008] Another imaging device consistent with the present inventionincludes a rotatable C-arm support having first and second ends. Thefirst end includes an x-ray source for initiating an imaging cycle andthe second end includes an x-ray receiving section positioned so thatx-rays emanating from the x-ray source enter the x-ray receivingsection. The x-ray receiving section generates an image representing theintensities of the x-rays entering the x-ray receiving section. Further,a calibration and tracking target is included and a tracking sensordetects the position, in three-dimensional space, of the calibration andtracking target; and a computer is coupled to the x-ray receivingsection and the tracking sensor. The computer detects motion of theC-arm based on changes in the position detected by the tracking sensor.

[0009] Another aspect consistent with the present invention is directedto a surgical instrument navigation system. The system comprises acomputer processor; a tracking sensor for sensing three-dimensionalposition information of a surgical instrument and transmitting theposition information to the computer processor; a memory coupled to thecomputer processor, the memory including computer instructions that whenexecuted by the computer processor cause the processor to generate anicon representing the surgical instrument and to overlay the icon on apre-acquired x-ray image, the icon of the surgical instrumentrepresenting the real-time position of the surgical instrument projectedinto the pre-acquired x-ray image and the icon being generated as afirst representation when the surgical instrument is positioned suchthat it is substantially viewable in the plane of the pre-acquired imageand the icon being generated as a second representation when thesurgical instrument is positioned such that it is substantiallyperpendicular to the plane of the pre-acquired image. Finally, a displayis coupled to the processor for displaying the generated iconsuperimposed on the pre-acquired image.

[0010] Yet another system consistent with the present inventioncomprises a computer processor and a tracking sensor for sensingthree-dimensional position information of a surgical instrument andtransmitting the position information to the computer processor. Amemory is coupled to the computer processor, the memory includingcomputer instructions that when executed by the computer processor causethe processor to generate an icon representing the surgical instrumentpositioned in a pre-acquired image of a patient's anatomy, the icon ofthe surgical instrument including a first portion corresponding to anactual position of the surgical instrument and a second portioncorresponding to a projection of the surgical instrument along a linegiven by a current trajectory of the surgical instrument. A display iscoupled to the processor for displaying the generated icon superimposedon the pre-acquired image.

[0011] Still further, another surgical instrument navigation systemconsistent with the present invention comprises a rotatable C-armincluding an x-ray source and an x-ray receiving section for acquiringx-ray images of a patient, the C-arm being rotatable about one of aplurality of mechanical axes. A computer processor is coupled to therotatable C-arm and a memory is coupled to the computer processor. Thememory stores the x-ray images acquired by the rotatable C-arm andcomputer instructions that when executed by the computer processor causethe computer processor to generate a line representing a projection of aplane parallel to one of the plurality of the mechanical axes of theC-arm into the x-ray image, the line enabling visual alignment of theone of the plurality of mechanical axes of the C-arm with an axisrelating complimentary image views. A display is coupled to theprocessor for displaying the generated line superimposed on the x-rayimage.

[0012] Yet another system consistent with the present invention is fordefining a surgical plan and comprises an x-ray imaging device; asurgical instrument; a tracking sensor for detecting the position, inthree-dimensional space, of the surgical instrument; a computerprocessor in communication with the tracking sensor for defining a pointin a virtual x-ray imaging path as the three-dimensional location of thesurgical instrument, the point being outside of a true x-ray imagingpath of the x-ray imaging device, the computer processor translatingposition of the surgical instrument within the virtual x-ray imagingpath to a corresponding position in the true x-ray imaging path; and adisplay coupled to the processor for displaying a pre-acquired x-rayimage overlaid with an iconic representation of the surgical instrument,the position of the iconic representation of the surgical instrument inthe pre-acquired x-ray image corresponding to the translated position ofthe surgical instrument.

[0013] Yet another system consistent with the present invention fordefining a surgical plan comprises a combination of elements. Theelements include an x-ray imaging device; a surgical instrument; atracking sensor for detecting the position, in three-dimensional space,of the surgical instrument; a computer processor in communication withthe tracking sensor for calculating a projection of the trajectory ofthe surgical instrument a distance ahead of the actual location of thesurgical instrument; and a display coupled to the processor fordisplaying a pre-acquired x-ray image overlaid with an iconicrepresentation of the surgical instrument and the calculated projectionof the trajectory of the surgical instrument.

[0014] Yet another system consistent with the present invention is foraligning a first bone segment with a second bone segment in a patient.The system comprises a first tracking marker attached to the first bonesegment and a second tracking marker attached to the second bonesegment. A tracking sensor detects the relative position, inthree-dimensional space, of the first and second tracking markers. Acomputer delineates boundaries of images of the first and second bonesegments in a pre-acquired x-ray image and when the second bone segmentis moved in the patient, the computer correspondingly moves thedelineated boundary of the second bone segment in the x-ray image. Adisplay is coupled to the computer and displays the pre-acquired x-rayimage overlaid with representations of the delineated boundaries of thefirst and second bone segments.

[0015] Yet another system consistent with the present invention isdirected to a system for placing a surgical implant into a patient. Thesystem comprises a computer processor; means for entering dimensions ofthe implant; a tracking sensor for sensing three-dimensional positioninformation of a surgical instrument on which the surgical implant isattached, the tracking sensor transmitting the position information tothe computer processor; and a memory coupled to the computer processor,the memory including computer instructions that when executed by thecomputer processor cause the processor to generate an icon representingthe surgical instrument and the attached surgical implant, and tooverlay the icon on a pre-acquired two-dimensional x-ray image, the iconof the surgical instrument representing the real-time position of thesurgical instrument relative to the pre-acquired two-dimensional x-rayimage.

[0016] In addition to the above mention devices and systems, theconcepts of the present invention may be practiced as a number ofrelated methods.

[0017] An additional method consistent with the present invention is amethod of acquiring a two-dimensional x-ray image of patient anatomyfrom a desired view direction. The method comprises generating thetwo-dimensional image using an x-ray imager; specifying a view directionin a three-dimensional image representing the patient anatomy;generating a two-dimensional digitally reconstructed radiograph (DRR)image based on the three-dimensional image and the specified viewdirection; and

[0018] determining that the two-dimensional x-ray image corresponds tothe desired view direction by matching the DRR image to the x-ray image.

[0019] Another method consistent with the present invention is a methodof calculating an angle between a surgical instrument and a planeselected in an x-ray image. The method comprises a number of steps,including: defining at least two points in the x-ray image; defining aplane passing through the x-ray image as the plane including the twopoints and linear projections of the two points as dictated by acalibration transformation used to calibrate the x-ray image for itsparticular imaging device; sensing a position of the surgical instrumentin three-dimensional space; and calculating the angle betweenintersection of a projection of the surgical instrument inthree-dimensional space and the plane.

[0020] Yet another method consistent with the present invention is amethod for aligning a fluoroscopic imager with a view direction of themedial axis of a patient's pedicle. The method comprises displaying athree-dimensional image of an axial cross-section of vertebra of thepatient; extracting an angle from the three-dimensional imagecorresponding to the angle separating an anterior/posterior axis and themedial axis of the pedicle; aligning the fluoroscopic imager with a longaxis of the patient; and rotating the fluoroscopic imager about the longaxis of the patient through the measured angle.

BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in andconstitute a part of this specification, illustrate several embodimentsconsistent with this invention and, together with the description, helpexplain the principles of the invention. In the drawings,

[0022]FIG. 1 is a diagram of an exemplary imaging system used to acquirex-ray images;

[0023]FIG. 2 is an image illustrating true and distorted images;

[0024]FIGS. 3A and 3B illustrate a projective transformation in afluoroscopic C-arm imager;

[0025]FIG. 4 is a flow chart illustrating methods consistent with thepresent invention for performing two-dimensional navigational guidance;

[0026]FIGS. 5A and 5B are exemplary fluoroscopic x-ray imagesillustrating the iconic graphical overlay of a surgical instrument;

[0027]FIG. 6 is a fluoroscopic image including a “cross hair” graphicaloverlay of an instrument;

[0028] FIGS. 7A-7C illustrate images of complementary views and an axisthat relates them;

[0029]FIG. 8 is an image of a lateral view of a patient's vertebraldisc;

[0030]FIG. 9 is an image of a lateral view of a spinal vertebra;

[0031]FIG. 10 is a diagram illustrating a system for specifying aplanned trajectory of a surgical instrument;

[0032]FIG. 11 is a flow chart illustrating a method for specifying aplanned trajectory of a surgical instrument;

[0033]FIGS. 12A through 12C are images of a fracture of a femurcontaining two bone fragments;

[0034]FIG. 13 is a flow chart illustrating methods for aligning bonefragments consistent with the present invention;

[0035]FIGS. 14A and 14B are images illustrating implantation of aninter-vertebral cage in the spine of a patient;

[0036]FIGS. 15A through 15C are images used in the acquisition of anx-ray view of the medial axis of a vertebral pedicle; and

[0037]FIGS. 16A and 16B are images used to illustrate the measurement ofout-of-plane angles based on fluoroscopic images.

DETAILED DESCRIPTION

[0038] As described herein, novel methods and systems improve surgicalnavigational guidance using one or more fluoroscopic x-ray images. Themethods and systems may be used for either navigational guidance usingonly two-dimensional fluoroscopic images or for navigational guidanceusing a combination of two-dimensional fluoroscopic images andthree-dimensional volumetric images, such as CT or MRI images.

[0039] Reference will now be made in detail to embodiments of theinvention, examples of which are illustrated in the accompanyingdrawings. Wherever possible, the same reference numbers will be usedthroughout the drawings to refer to the same or like parts.

System Overview

[0040]FIG. 1 is a diagram of an exemplary imaging system used to acquirex-ray images. Fluoroscopic imaging device 100 is a fluoroscopic C-armx-ray imaging device that includes C-arm 103, x-ray source 104, x-rayreceiving section 105, a calibration and tracking target 106, andradiation sensors 107. Calibration and tracking target 106 includesinfrared reflectors (or alternatively infrared emitters) 109 andcalibration markers 111. C-arm control computer 115 allows a physicianto control the operation of imaging device 100, such as setting imagingparameters.

[0041] One appropriate implementation of imaging device 100 is the“Series9600 Mobile Digital Imaging System,” from OEC Medical Systems,Inc., of Salt Lake City, Utah, although calibration and tracking target106 and radiation sensors 107 are typically not included in theSeries9600 Mobile Digital Imaging System and may have to be added. The“Series9600 Mobile Digital Imaging System” is otherwise structurallysimilar to imaging system 100.

[0042] In operation, x-ray source 104 generates x-rays that propagatethrough patient 110 and calibration target 106, and into x-ray receivingsection 105. Receiving section 105 generates an image representing theintensities of the received x-rays. Typically, receiving section 105comprises an image intensifier that converts the x-rays to visible lightand a charge coupled device (CCD) video camera that converts the visiblelight to digital images. Receiving section 105 may also be a device thatconverts x-rays directly to digital images, thus potentially avoidingdistortion introduced by first converting to visible light.

[0043] Fluoroscopic images taken by imaging device 100 are transmittedto computer 115, where they may further be forwarded to computer 120.Computer 120 provides facilities for displaying (on monitor 121),saving, digitally manipulating, or printing a hard copy of the receivedimages. Three-dimensional images, such as pre-acquired patient specificCT/MR data set 124 or a three-dimensional atlas data set 126 (describedin more detail below) may also be manipulated by computer 120 anddisplayed by monitor 121. Images, instead of or in addition to beingdisplayed on monitor 121, may also be displayed to the physician througha heads-up-display.

[0044] Although computers 115 and 120 are shown as two separatecomputers, they alternatively could be variously implemented as multiplecomputers or as a single computer that performs the functions performedby computers 115 and 120. In this case, the single computer wouldreceive input from both C-arm imager 100 and tracking sensor 130.

[0045] Radiation sensors 107 sense the presence of radiation, which isused to determine whether or not imaging device 100 is actively imaging.The result of their detection is transmitted to processing computer 120.Alternatively, a person may manually indicate when device 100 isactively imaging or this function can be built into x-ray source 104,x-ray receiving section 105, or control computer 115.

[0046] In operation, the patient is positioned between the x-ray source104 and the x-ray receiving section 105. In response to an operator'scommand input at control computer 115, x-rays emanate from source 104and pass through patient 110, calibration target 106, and into receivingsection 105, which generates a two-dimensional image of the patient.

[0047] C-arm 103 is capable of rotating relative to patient 110,allowing images of patient 110 to be taken from multiple directions. Forexample, the physician may rotate C-arm 103 in the direction of arrows108 or about the long axis of the patient. Each of these directions ofmovement involves rotation about a mechanical axis of the C-arm. In thisexample, the long axis of the patient is aligned with the mechanicalaxis of the C-arm.

[0048] Raw images generated by receiving section 105 tend to suffer fromundesirable distortion caused by a number of factors, including inherentimage distortion in the image intensifier and external electromagneticfields. Drawings representing ideal and distorted images are shown inFIG. 2. Checkerboard 202 represents the ideal image of a checkerboardshaped object. The image taken by receiving section 105, however, cansuffer significant distortion, as illustrated by distorted image 204.

[0049] The image formation process in a system such as fluoroscopicC-arm imager 100 is governed by a geometric projective transformationwhich maps lines in the fluoroscope's field of view to points in theimage (i.e., within the x-ray receiving section 105). This concept isillustrated in FIGS. 3A and 3B. Image 300 (and any image generated bythe fluoroscope) is composed of discrete picture elements (pixels), anexample of which is labeled as 302. Every pixel within image 300 has acorresponding three-dimensional line in the fluoroscope's field of view.For example, the line corresponding to pixel 302 is labeled as 304. Thecomplete mapping between image pixels and corresponding lines governsprojection of objects within the field of view into the image. Theintensity value at pixel 302 is determined by the densities of theobject elements (i.e., portions of a patient's anatomy, operating roomtable, etc.) intersected by the line 304. For the purposes of computerassisted navigational guidance, it is necessary to estimate theprojective transformation which maps lines in the field of view topixels in the image, and vice versa. Geometric projective transformationis well known in the art.

[0050] Intrinsic calibration, which is the process of correcting imagedistortion in a received image and establishing the projectivetransformation for that image, involves placing “calibration markers” inthe path of the x-ray, where a calibration marker is an object opaque orsemi-opaque to x-rays. Calibration markers 111 are rigidly arranged inpredetermined patterns in one or more planes in the path of the x-raysand are visible in the recorded images. Tracking targets, such asemitters or reflectors 109, are fixed in a rigid and known positionrelative to calibration markers 111.

[0051] Because the true relative position of the calibration markers 111in the recorded images are known, computer 120 is able to calculate anamount of distortion at each pixel in the image (where a pixel is asingle point in the image). Accordingly, computer 120 can digitallycompensate for the distortion in the image and generate adistortion-free, or at least a distortion improved image. Alternatively,distortion may be left in the image, and subsequent operations on theimage, such as superimposing an iconic representation of a surgicalinstrument on the image (described in more detail below), may bedistorted to match the image distortion determined by the calibrationmarkers. The same calibration markers can also be used to estimate thegeometric perspective transformation, since the position of thesemarkers are known with respect to the tracking target emitters orreflectors 109 and ultimately with respect to tracking sensor 130. Amore detailed explanation of methods for performing intrinsiccalibration is described in the references B. Schuele et al.,“Correction of Image Intensifier Distortion for Three-DimensionalReconstruction,” presented at SPIE Medical Imaging 1995, San Diego,Calif., 1995 and G. Champleboux et al., “Accurate Calibration of Camerasand Range Imaging Sensors: the NPBS Method,” Proceedings of the 1992IEEE International Conference on Robotics and Automation, Nice, France,May 1992, and U.S. application Ser. No. 09/106,109, filed on Jun. 29,1998 by the present assignee, the contents of which are herebyincorporated by reference.

[0052] Calibration and tracking target 106 may be attached to x-rayreceiving section 105 of the C-arm. Alternately, the target 106 can bemechanically independent of the C-arm, in which case it should bepositioned such that the included calibration markers 111 are visible ineach fluoroscopic image to be used in navigational guidance. Element 106serves two functions. The first, as described above, is holdingcalibration markers 111 used in intrinsic calibration. The secondfunction, which is described in more detail below, is holding infraredemitters or reflectors 109, which act as a tracking target for trackingsensor 130.

[0053] Tracking sensor 130 is a real-time infrared tracking sensorlinked to computer 120. Specially constructed surgical instruments andother markers in the field of tracking sensor 130 can be detected andlocated in three-dimensional space. For example, a surgical instrument140, such as a drill, is embedded with infrared emitters or reflectors141 on its handle. Tracking sensor 130 detects the presence and locationof infrared emitters or reflectors 141. Because the relative spatiallocations of the emitters or reflectors in instrument 140 are known apriori, tracking sensor 130 and computer 120 are able to locateinstrument 140 in three-dimensional space using well known mathematicaltransformations. Instead of using infrared tracking sensor 130 andcorresponding infrared emitters or reflectors, other types of positionallocation devices are known in the art, and may be used. For example, apositional location device may also be based on magnetic fields, sonicemissions, or radio waves.

[0054] Reference frame marker 150, like surgical instrument 140, isembedded with infrared emitters or reflectors, labeled 151. As withinstrument 140, tracking sensor 130 similarly detects the spatiallocation of emitters/reflectors 151, through which tracking sensor 130and computer 120 determine the three-dimensional position of dynamicreference frame marker 150. The determination of the three-dimensionalposition of an object relative to a patient is known in the art, and isdiscussed, for example, in the following references, each of which ishereby incorporated by reference: PCT Publication WO 96/11624 to Bucholzet al., published Apr. 25, 1996; U.S. Pat. No. 5,384,454 to Bucholz;U.S. Pat. No. 5,851,183 to Bucholz; and U.S. Pat. No. 5,871,445 toBucholz.

[0055] During an operation, dynamic reference frame marker 150 isattached in a fixed position relative to the portion of the patient tobe operated on. For example, when inserting a screw into the spine ofpatient 110, dynamic reference frame marker 150 may be physicallyattached to a portion of the spine of the patient. Because dynamicreference frame 150 is in a fixed position relative to the patientanatomy, and instrument 140 can be accurately located in threedimensional space relative to dynamic reference frame 150, instrument140 can also be located relative to the patient's anatomy.

[0056] As discussed above, calibration and tracking target 106 alsoincludes infrared emitters or reflectors 109 similar to those ininstrument 140 or dynamic reference frame 150. Accordingly, trackingsensor 130 and computer 120 may determine the three-dimensional positionof calibration target 106 relative to instrument 140 and/or dynamicreference frame 150 and thus the patient position.

System Operation

[0057] In general, the imaging system shown in FIG. 1 assists physiciansperforming surgery by displaying real-time or pre-acquired images, suchas fluoroscopic x-ray images, of patient 110 on display 121.Representations of surgical instruments 140 are overlaid on pre-acquiredfluoroscopic images of patient 110 based on the position of theinstruments determined by tracking sensor 130. In this manner, thephysician is able to see the location of the instrument relative to thepatient's anatomy, without the need to acquire real-time fluoroscopicimages, thus greatly reducing radiation exposure to the patient and tothe surgical team. “Pre-acquired,” as used herein, is not intended toimply any required minimum duration between receipt of the x-ray signalsand displaying the corresponding image. Momentarily storing thecorresponding digital signal in computer memory while displaying thefluoroscopic image constitutes pre-acquiring the image.

[0058]FIG. 4 is a flow chart illustrating methods consistent with thepresent invention for performing two-dimensional navigational guidanceusing the system of FIG. 1. The physician begins by acquiring one ormore fluoroscopic x-ray images of patient 110 using imager 100 (step400). As previously mentioned, acquiring an x-ray image triggersradiation sensors 107, which informs computer 120 of the beginning andend of the radiation cycle used to generate the image. For afluoroscopic x-ray image acquired with imager 100 to be useable fornavigational guidance, imager 100, when acquiring the image, should bestationary with respect to patient 110. If C-arm 103 or patient 110 ismoving during image acquisition, the position of the fluoroscope willnot be accurately determined relative to the patient's reference frame.Thus, it is important that the recorded position of imager 100 reflectsthe true position of the imager at the time of image acquisition. Ifimager 100 moves during the image acquisition process, or if imager 100moves after image acquisition but before its position is recorded, thecalibration will be erroneous, thus resulting in incorrect graphicaloverlays. To prevent this type of erroneous image, computer 120 mayexamine the position information from tracking sensor 130 whileradiation sensors 107 are signaling radiation detection. If thecalibration and tracking target 106 moves relative to dynamic referenceframe 150 during image acquisition, this image is marked as erroneous.(Steps 401 and 402).

[0059] At the end of the radiation cycle, computer 120 retrieves theacquired image from C-arm control computer 115 and retrieves thelocation information of target marker 106 and dynamic reference frame150 from tracking sensor 130. Computer 120 calibrates the acquiredimage, as described above, to learn its projective transformation andoptionally to correct distortion in the image, (step 1403), and thenstores the image along with its positional information (step 404). Theprocess of steps 400-404 is repeated for each image that is to beacquired (step 405).

[0060] Because the acquired images are stored with the positionalinformation of the calibration and tracking target 106 and dynamicreference frame 150, the position of C-arm 103, x-ray source 104, andreceiving section 105 for each image, relative to patient 110, can becomputed based upon the projective transformation identified in thecalibration process. During surgery, tracking sensor 130 and computer120 detect the position of instrument 140 relative to dynamic referenceframe 150, and hence relative to patient 110. With this information,computer 120 dynamically calculates, in real-time, the projection ofinstrument 140 into each fluoroscopic image as the instrument is movedby the physician. A graphical representation of instrument 140 may thenbe overlaid on the fluoroscopic images (step 406). The graphicalrepresentation of instrument 140 is an iconic representation of wherethe actual surgical instrument would appear within the acquiredfluoroscopic x-ray image if imager 100 was continuously acquiring newimages from the same view as the original image. There is no theoreticallimit to the number of fluoroscopic images on which the graphicalrepresentations of instrument 140 may be simultaneously overlaid.

[0061]FIGS. 5A and 5B are exemplary fluoroscopic x-ray imagesillustrating the iconic graphical overlay of a surgical instrument.Fluoroscopic image 500, shown in FIG. 5A, is an image of a lateral viewof the lumbar spine. Graphical overlay 502 is the iconic overlay of asurgical instrument, such as a drill, within image 500. As the physicianmoves the drill, computer 120 recalculates and displays the new locationof graphical overlay 502. The diamond shaped end of overlay 502, labeledas area 503, represents the tip of the instrument. The physician can useimage 500 and overlay 502 to visualize the position and orientation ofthe surgical tool relative to the patient's anatomy.

[0062] In certain situations, the physician may wish to know where thetip of the instrument would be if the instrument were projected along aline given by the instrument's current trajectory. Consistent with anaspect of the present invention, at the physician's command, computer120 may calculate and display this projection. Area 505 in FIG. 5B is aprojection of graphical overlay 502. As shown, the “look-ahead”trajectory 505 of overlay 502 is displayed in a different line stylethan overlay 502. Computer 120 may vary the length of look-aheadtrajectory 505 as directed by the physician through a suitable computerinterface device, such as a keypad, mouse, or touch pad. In this manner,computer 120 assists the physician in visualizing where the instrumentwould be in the patient if it were advanced a predetermined distance inthe patient.

[0063] Although the “look-ahead” technique described above projected thegraphical representation of the instrument into the image, there is norequirement that the instrument's graphical representation be in thespace of the image for look-ahead trajectory 505 to be projected intothe image. For example, the physician may be holding the instrumentabove the patient and outside the space of the image, so that therepresentation of the instrument does not appear in the image. However,it may still be desirable to project look-ahead portion 505 into theimage to facilitate planning of a surgical procedure.

[0064] When surgical instrument 140 is perpendicular to the plane of thefluoroscopic image, the graphical overlay of the surgical instrumentessentially collapses to a point, making it difficult to view. Toalleviate this problem, computer 120 may optionally use a differentgraphical representation of instrument 140 when the distance in theimage plane between the tip and the tail of instrument 140 becomessmaller than a fixed distance (e.g., 15 pixels).

[0065]FIG. 6 is a fluoroscopic image including graphical overlay 601 ofinstrument 140, including a small “cross hair image” representing tip602 and a larger cross hair representing tail 603 of instrument 601.Computer 120 automatically switches between the cross hairrepresentation shown in FIG. 6 and the “straight line” representationshown in FIG. 5.

[0066] Frequently, the physician would like to acquire two complementaryfluoroscopic images of the patient, such as images from ananterior/posterior view and a lateral view of the vertebral discs. Thecomplementary views are related to one another by a rotation about anaxis by a particular amount. For example, an anterior/posterior view isrelated to a lateral view by a 90 degree rotation around the axisrunning parallel through the length of the patient. When the mechanicalaxis of rotation of C-arm 103 is aligned with the axis relating thecomplementary views (e.g., when the mechanical axis is aligned with theaxis running through the length of the patient), the physician canaccurately and quickly switch between the complementary views by simplyrotating C-arm 103 through the separation of the complementary views(usually 90 degrees). Generally, however, the axis of rotation of C-arm103 is not inherently aligned with the axis that relates thecomplementary views, requiring the physician to perform a series of timeconsuming trial-and-error based adjustments of the fluoroscope'sposition through two or more axes of rotation.

[0067] Consistent with an aspect of the present invention, software oncomputer 120 allows the surgeon to easily adjust the fluoroscope'sposition so that one of its mechanical rotation axes, such as the axisof rotation shown by arrows 108 in FIG. 1, is aligned with the axis ofrotation relating the complementary views. The surgeon may then acquirethe second image in the complementary image set simply by rotating C-arm103 a certain amount, typically 90 degrees, about the aligned axis.

[0068] Images of complementary views and the axis that relates them areillustrated in FIGS. 7A-7C. The image of FIG. 7A illustrates a lateralview of the patient's vertebral disc, in which the view direction (i.e.,the direction of the central ray of fluoroscopic imager 100) isapproximately parallel to the two vertebral end plates, labeled asendplate 705 and endplate 706. Line 702 is the projection of the planesubstantially parallel to end plates 705 and 706. Similarly, the imageshown in FIG. 7B is an anterior/posterior view of the patient'svertebral disc, in which the view direction is parallel to plane 702.The axis of rotation 704 that spatially relates the image view of FIG.7A and the image view of FIG. 7B is a line perpendicular to plane 702.That is, rotating the image view of FIG. 7A ninety degrees about theline perpendicular to plane 702 will result in the image view shown inFIG. 7B. FIG. 7C is a three-dimensional representation of the anatomyshown in FIGS. 7A and 7B. The line perpendicular to plane 702 is shownby axis of rotation 704.

[0069]FIG. 8 is an image of a lateral view of the patient's vertebraldisc, similar to FIG. 7A. In FIG. 8, however, computer 120 has drawnline 802, which represents the projection of a plane that isperpendicular to one of the C-arm's mechanical axes. Line 804 representsthe plane that spatially relates the complementary views. With line 802visible, the physician may adjust the position of fluoroscopic imager100 so that line 802 is lined up with line 804. At this point, switchingbetween the complementary views simply involves rotating C-arm 103 abouta single mechanical axis.

[0070] Although the alignment of lines 802 and 804, as discussed above,was illustrated using both lines 802 and 804 drawn on the fluoroscopicimage, in practice, it may only be necessary to display line 802 in theimage. In this case, line 804 is mentally visualized by the physician.Additionally, although the relation of complimentary views was discussedusing the example of the spine, complimentary fluoroscopic images ofother anatomical regions, such as, for example, the pelvis, femur, orcranium, may similarly be obtained by application of the above discussedconcepts.

[0071] Before, or during, surgery, the physician may find it desirableto input an operation “plan” to computer 120. The plan may, for example,specify a desired trajectory of a surgical instrument superimposed on afluoroscopic image. During the surgical navigation process, the goal ofthe surgeon would be to align the graphical icon representing thereal-time location of the surgical instrument with the graphical overlayrepresenting the planned trajectory.

[0072]FIG. 9 is an image of a lateral view of a spinal vertebra. Assumethe goal of the operation plan is to define a line that passes along adesired trajectory within the image of the vertebra. One method ofaccomplishing this goal is to directly input the desired trajectoryinformation to computer 120 using traditional computer input devices.While this method of directly interacting with computer 120 is possible,it can be cumbersome and disruptive during surgery. Consistent with anaspect of the present invention, an alternative method of accomplishingthis is for the physician to position the surgical instrument on thesurface of the bone or skin in the desired orientation, and then projectthe tip of the instrument forward using the previously describedlook-ahead technique. More specifically, the desired trajectory isspecified by (1) adjusting the position and orientation of theinstrument near the patient with virtual look-ahead active, and (2)adjusting the length of the virtual look-ahead. FIG. 9 illustrates theiconic representation of instrument 901 and the virtual look-aheadprojection of the instrument 902. Once the desired trajectory isachieved, the surgeon may direct computer 120 to “freeze” the plannedtrajectory on display 121. The desired trajectory can be obtained usingone or more C-arm fluoroscopic images with two or more being required todefine a specific three-dimensional trajectory which can then bedisplayed on any C-arm fluoroscopic view. The freeze operation may beinput to computer 120 through, for example, a simple input device suchas a foot pedal. The physician may then proceed with the operation,using the overlay of the planned target as a guide.

[0073] Yet another method consistent with the present invention forspecifying a planned trajectory of a surgical instrument, which, unlikethe method discussed above, does not require positioning the surgicalinstrument on or near the patient's bone, is illustrated in FIGS. 10 and11.

[0074] As shown in FIG. 10, during the acquisition of an image, patient1001 is positioned between C-arm x-ray source 1004 and x-ray receivingsection 1006. Fluoroscopic images of patient 1001 are created by thex-rays emitted from x-ray source 1004 as they travel in the pathgenerally outlined by cone 1010 through patient 1001. Line 1011, in thecenter of cone 1010, corresponds to the origin (i.e., the center point)in the acquired fluoroscopic images. Physician 1020, standing within therange of tracking sensor 1030, but away from patient 1001, commands thecomputer to create an explicit correspondence between the fluoroscope'simaging cone 1010 and a “virtual” cone 1012 at an arbitrary position inspace (which is visible to the tracking sensor). Once this virtual conehas been defined, the surgical instrument 1040 can be projected fromthis virtual cone into one or more pre-acquired fluoroscopic images inthe same manner as if the instrument were located in the actual cone1010 corresponding to a given image. In this manner, physician 1020 canplan the trajectory of surgical instrument 1040 by simply moving theinstrument in the coordinate system established by the virtual cone.

[0075] To define the correspondence between actual and virtual cones, itis necessary for the physician to define the position of the virtualcone relative to the tracking sensor. In general, there are many ways todefine a cone in space. For example, the position and orientation of acone can be defined by three points, one corresponding to its apex, onecorresponding to a second point along its central axis, and a thirdcorresponding to the rotation of the cone about the central axis.Therefore, one way to define the cone would be to use the tip of thesurgical instrument to define these three points in space relative tothe tracking sensor. Another way to define this correspondence is to usea single measurement of a surgical instrument. Using this method, theaxis of the instrument corresponds to the axis of the cone, the tip ofthe instrument corresponds to a fixed point along the axis of the cone(which could be the apex, but could also be another point along thecentral axis), and the orientation of the instrument about its axiscorresponds to the orientation of the cone about its axis. In generalany set of measurements which define the position and orientation of agiven cone can be used to establish the correspondence between theactual and virtual cones.

[0076] The operations illustrated in FIG. 10 are shown in the flowchartof FIG. 11. To begin, the physician holds the surgical instrument 1040in the position that defines the virtual cone in the manner as outlinedin the previous paragraph (step 1101). Computer 120 locates the positionof instrument 1040, which effectively corresponds the position andorientation of the virtual cone to the actual cone (step 1102). Computer120 projects additional movements of instrument 1040 into one or morepreviously acquired fluoroscopic images as if the instrument were beingmoved in the actual cone corresponding to a given image (step 1103). Inthis manner, the physician can align the instrument to particular pointsor trajectories within previously acquired images. At the physician'scommand, computer 120 “freezes” the position and/or orientation of theinstrument in the displayed fluoroscopic image(s) and uses those forsubsequent processing and plan generation (step 1104).

[0077] It is also consistent with this invention to provide automatedplanning using computer analysis techniques to define an “optimal”trajectory in the C-arm images. Once the optimal trajectory isdetermined, computer 120 overlays the optimal trajectory in thefluoroscopic image. For example, automated plans can be generated usingcomputational techniques to reduce a specified amount of lordosis inspine surgery.

Alignment of Bone Fragments

[0078] A common clinical problem, especially in orthopaedic trauma, isthe realignment (reduction) of broken or misaligned bone fragments. FIG.12A is a fluoroscopic image of a fracture of the femur containing twobone fragments 1201 and 1202. The physician's job is to realign the bonefragments so that the femur can properly heal.

[0079]FIG. 13 is a flow chart illustrating methods for aligning bonefragments consistent with the present invention. In general, one of bonefragments 1201 or 1202 is used as a fixed reference frame and the otheras a dynamic reference frame. When the physician moves the bone fragmentcorresponding to the dynamic reference frame, tracking sensor 130detects the movement and updates the x-ray image to reflect the newlocation of the bone fragment in the patient.

[0080] To begin the alignment procedure, the physician places a trackingsensor marker on each of bone fragments 1201 and 1202 (step 1301) andacquires the fluoroscopic images, (step 1302), such as the image shownin FIG. 12A. Computer 120 processes the acquired image to obtainpositional location information and to calibrate the image (step 1303,this step is identical to steps 401-403 in FIG. 4).

[0081] After acquisition of the fluoroscopic image(s), computer 120 usesimage detection and extraction techniques to delineate the boundaries ofthe bone fragments in the images (step 1304). Suitable edge detectionalgorithms for generating the contours are well known in the art, andmay be, for example, the Canny edge detector, the Shen-Casten edgedetector, or the Sobel edge detector. An edge detected version of FIG.12A is shown in FIG. 12B, in which the resulting contour correspondingto bone fragment 1201 is labeled as 1203 and the contour correspondingto bone fragment 1202 is labeled as 1204. Contours 1203 and 1204 may be,as shown in FIG. 12B, graphically superimposed by computer 120 on theacquired image(s).

[0082] Overlaying the detected image contours on the fluoroscopic imageallows the physician to easily identify the correspondence between imagecontours 1203-1204 and bone fragments 1201-1202. The physician inputsthis correspondence into computer 120 (step 1305). Alternatively,computer 120 may automatically identify the correspondence between theimage contours and the bone fragments. Once the correspondence isestablished, the physician specifies which contour is to remain fixedand which is to be repositioned. The tracking sensor marker attached tothe fragment to be repositioned is referred to as the dynamic referencemarker and the tracking sensor marker attached to the fixed fragment isreferred to as the fixed reference frame marker, although physically thedynamic reference marker and the fixed reference frame marker may beidentical.

[0083] During surgical navigation, the physician moves the bone fragmenthaving the dynamic reference marker (step 1306). Tracking sensor 130detects the position of the dynamic reference frame marker and the fixedframe marker. With this information and the previously generatedpositional location information, computer 120 calculates and displaysthe new position of the dynamic reference frame, and hence itscorresponding bone fragment, in the fluoroscopic image (step 1307). FIG.12C illustrates an updated version of the fluoroscopic image contour1203 corresponding to the fixed bone fragment and contour 1204corresponding to the new location of the dynamic reference marker andits bone fragment.

[0084] Methods described above for aligning bone fragments may also beapplied to the proper alignment of multiple vertebral bodies, forexample in the reduction of scoliosis.

Three-Dimensional Images

[0085] The navigational guidance system consistent with the presentinvention is not limited to providing surgical navigational guidancewith two-dimensional fluoroscopic images. Three-dimensional volumetricdata sets may also be overlaid with graphical representations of asurgical instrument. Three-dimensional data sets (such as CT or MRI) maybe either pre-acquired or acquired during the operation.

[0086] Two types of three-dimensional data sets are typically used insurgical navigation: patient-specific image data and non-patientspecific or atlas data. Patient-specific three-dimensional images aretypically acquired prior to surgery using computed tomography (CT),magnetic resonance (MR), or other known three-dimensional imagingmodalities, although intra-operative acquisition is also possible. Atlasdata is non-patient specific three-dimensional data describing a“generic” patient. Atlas data may be acquired using CT, MR or otherimaging modalities from a particular patient; and may even compriseimages from several modalities which are spatially registered (e.g., CTand MR together in a common coordinate system). Atlas data may beannotated with supplemental information describing anatomy, physiology,pathology, or “optimal” planning information (for example screwplacements, lordosis angles, scoliotic correction plans, etc).

[0087] A three-dimensional patient CT or MR data set is shown in FIG. 1as data set 124 and atlas data is illustrated in FIG. 1 as data set 126.

[0088] Before overlaying a three-dimensional image with graphicalrepresentations of surgical instruments, the correspondence betweenpoints in the three-dimensional image and points in the patient'sreference frame must be determined. This procedure is known asregistration of the image. One method for performing image registrationis described in the previously mentioned publications to Bucholz.Three-dimensional patient specific images can be registered to a patienton the operating room table (surgical space) using multipletwo-dimensional image projections. This process, which is often referredto as 2D/3D registration, uses two spatial transformations that can beestablished. The first transformation is between the acquiredfluoroscopic images and the three-dimensional image data set (e.g., CTor MR) corresponding to the same patient. The second transformation isbetween the coordinate system of the fluoroscopic images and anexternally measurable reference system attached to the fluoroscopicimager. Once these transformations have been established, it is possibleto directly relate surgical space to three-dimensional image space.

[0089] When performing three-dimensional registration, as withtwo-dimensional registration, imager 100, when acquiring the image,should be stationary with respect to patient 110. If C-arm 103 orpatient 110 is moving during image acquisition, the position of thefluoroscope will not be accurately determined relative to the patient'sreference frame. Accordingly, the previously described technique fordetecting movement of imager 100 during the image acquisition processcan be used when acquiring fluoroscopic images that are to be used in2D/3D registration. That is, as described, computer 120 may examine theposition information from tracking sensor 130 while radiation sensors107 are signaling radiation detection. If the calibration and trackingtarget 106 moves relative to dynamic reference frame 150 during imageacquisition, this image is marked as erroneous.

[0090] It may be necessary to acquire complementary fluoroscopic views(e.g., lateral and anterior/posterior) to facilitate 2D/3D registration.The techniques previously discussed in reference to FIGS. 7-8 andrelating to the acquisition of complementary views can be applied here.

[0091] Once registered, computer 120 may use positional information ofinstrument 140 to overlay graphical representations of the instrument inthe three-dimensional image as well as the two-dimensional fluoroscopicimages.

Hybrid Use of Three-Dimensional and Two-Dimensional Image Data

[0092] The two-dimensional images generated by imager 100 are not alwaysable to adequately represent the patient's bone structure. For example,fluoroscopic x-ray images are not effective when taken through thelength of the patient (i.e., from the point of view looking down at thepatient's head or up from the patient's feet) because the large numberof bones that the x-rays pass through occlude one another in the finalimage. However, information required for planning a surgical procedurewhich is not otherwise available based on two-dimensional image dataalone may be extracted from a three-dimensional image data set such as aCT or MR image data set. The extracted information may then betransferred to the two-dimensional x-ray images generated by imager 100and used in surgical navigation. The following examples describeadditional methods for using three-dimensional and two-dimensional datain surgical navigation.

EXAMPLE 1 Importing Three-Dimensional Surgical Implant Specifications toTwo-Dimensional Images

[0093]FIGS. 14A and 14B are images illustrating the implantation of aninter-vertebral cage in the spine of a patient. An inter-vertebral cageis a roughly cylindrical spinal implant that is inserted in the discspace between adjacent spinal vertebrae. The physician may find itdifficult, if not impossible, to choose the appropriate length of aninter-vertebral cage based upon two-dimensional images such as the imageof FIG. 14A.

[0094] Rectangle 1401 represents the projection of the cylindricalinter-vertebral cage into the image. While the long axis of the cylinderappears to be completely within the bone in this image, this may not bethe case due to curvature of the anterior aspect of vertebrae 1402. FIG.14B is an image of a three-dimensional axial CT cross section of thevertebrae. Corner 1403 of rectangle 1401 protrudes from the bone—ahighly undesirable situation that cannot be reliably detected in x-rayimages such as that of FIG. 14A. Accordingly, when faced with thissituation, the appropriate cage length should be chosen based upon oneor more axial CT images, such as that in FIG. 14B. Selection of the cagelength can be performed automatically by computer 120 orsemi-automatically with the input of the physician.

[0095] Once the cage length has been determined by the physician andentered into computer 120, the length value can then be used by computer120 in properly displaying the graphical overlay in the associatedtwo-dimensional image. The position of the surgical instrument used tohold the cage during the insertion process, as detected by trackingsensor 130, is used to calculate the position of the cage in FIG. 14Aduring the two-dimensional navigational process.

[0096] Although the above discussed example was with a cylindricalspinal implant, in general, the described concepts could be applied toany surgical implant.

EXAMPLE 2 Acquisition of an X-Ray View Down the Medial Axis of aVertebral Pedicle

[0097] In certain clinical procedures, it may be desirable to acquire afluoroscopic x-ray image view looking substantially straight down themedial axis of a vertebral pedicle. For the purposes of this example, avertebral pedicle can be thought of as a cylinder, and the medial axiscorresponds to the central axis of the cylinder.

[0098]FIG. 15A is an x-ray image in which the view direction of theimager is aligned with the medial axis of the pedicle (i.e., the medialaxis of the pedicle is into the plane of the image). In this so-called“owl's eye” view, the pedicle appears as circle 1501 within the image.It is often difficult to precisely acquire this view using onlyfluoroscopic x-ray images, as it is difficult to align the viewdirection of imager 100 with the medial axis of the pedicle using onlyfluoroscopic images.

[0099] Given an anterior/posterior fluoroscopic image view of the spine,such as the one shown in FIG. 15B, and given that the mechanical axis ofthe fluoroscope is aligned with the patient's long axis (i.e., axis 704in FIG. 7C), an axial CT cross section of a vertebra can be used toquickly and easily acquire a high quality owl's eye view, such as theview of FIG. 15A.

[0100]FIG. 15C is an image of an axial CT cross section of a vertebra.With this image, computer 120 or the physician may measure angle 1504between the anterior/posterior axis 1502 and the projection of themedial axis 1503 of the pedicle 1501 into the axial plane. The physicianmay then rotate imager 100 by the measured angle about the mechanicalrotation axis that is aligned with the patient's long axis 704. Becausemost fluoroscopic imagers, such as imager 100, have angle indicators,rotation by the desired amount is trivial. However, if the physicianrequires additional accuracy in the rotation, tracking sensor 130,because it detects the position of C-arm 103, can be used to moreprecisely measure the rotation angle.

EXAMPLE 3 Use of Digitally Reconstructed Radiography in the Placement ofa Surgical Implant

[0101] With conventional fluoroscopic x-ray image acquisition, radiationpasses through a physical media to create a projection image on aradiation sensitive film or an electronic image intensifier. Given a 3DCT data set, a simulated x-ray image can also be generated using atechnique known as digitally reconstructed radiography (DRR). DRR iswell known in the art, and is described, for example, by L. Lemieux etal., “A Patient-to-Computed-Tomography Image Registration Method Basedon Digitally Reconstructed Radiographs,” Medical Physics 21(11), pp1749-1760, November 1994.

[0102] When a DRR image is created, a fluoroscopic image is formed bycomputationally projecting volume elements (voxels) of the 3D CT dataset onto a selected image plane. Using a 3D CT data set of a givenpatient, it is possible to create a DRR image that appears very similarto a corresponding x-ray image of the same patient. A requirement forthis similarity is that the “computational x-ray imager” and actualx-ray imager use similar intrinsic imaging parameters (e.g., projectiontransformations, distortion correction) and extrinsic imaging parameters(e.g., view direction). The intrinsic imaging parameters can be derivedfrom the calibration process.

[0103] A DRR image may be used to provide guidance to the surgeon in theproblem discussed in Example 1 of appropriately placing aninter-vertebral cage in the patient. Given a 3D CT data set of twoadjacent vertebrae, the physician, interacting with computer 120, maymanually position a 3D CAD model of an inter-vertebral cage in aclinically desired position in the three-dimensional view of thevertebrae. The physician may then use the DRR technique to synthesize ananterior/posterior, lateral, or other x-ray view of the vertebraeshowing the three-dimensional CAD model of the inter-vertebral cage.Thus, a synthetic fluoroscopic x-ray image can be created whichsimulates what a properly placed cage would look like afterimplantation.

[0104] The simulated x-ray images may be compared to the actual imagestaken by imager 100 during surgery. The goal of the surgeon is toposition the implant such that the intra-operative images match the DRRimages. For this comparison, two types of intra-operative images maypreferably be used. First, conventional fluoroscopy could be used toacquire an image after the inter-vertebral cage has been implanted.Second, images acquired prior to cage placement could be supplementedwith superimposed graphical icons representing the measured cageposition. In either case, the synthetic fluoroscopic image can be usedas a template to help guide the surgeon in properly placing theinter-vertebral cage.

[0105] Although the above example was described in the context ofimplanting an inter-vertebral cage, implants other than theinter-vertebral cage could also be used.

EXAMPLE 4 Obtaining a Particular Two-Dimensional View Direction UsingDigitally Reconstructed Radiograph Images

[0106] The DRR technique can be used to provide guidance to thephysician when acquiring an owl's eye view of a vertebral pedicle. Givena three-dimensional CT data set containing a vertebra and associatedpedicle, the physician may use computer 120 to manually locate athree-dimensional representation of the pedicle's medial axis relativeto the three-dimensional images of the vertebrae. Once this placementhas been achieved, it is possible to synthesize an owl's eye view of thevertebrae based upon the view direction specified by the physician'sselection of the three-dimensional medial axis. This synthetic image canthen be displayed to the surgeon during surgery and used to guide theacquisition of an actual owl's eye view using the fluoroscope. Byvisually comparing fluoroscopic images taken while positioning thefluoroscope to the synthetic owl's eye view, the physician can acquire atrue fluoroscopic image with a view direction approximately equal to themanually selected medial axis. In this manner, a high quality owl's eyeview can be acquired.

[0107] Although the above example was described in the context ofsynthesizing a two-dimensional owl's eye view, in general, anythree-dimensional view direction can be selected and a correspondingtwo-dimensional image synthesized and used to acquire a fluoroscopictwo-dimensional image.

EXAMPLE 5 Measuring Out-Of-Plane Angles Based On Fluoroscopic Images

[0108] It may be desirable to measure the angle between the trajectoryof a surgical instrument and the plane of a fluoroscopic image (such asa plane aligned with the mid-line of the spine 1502) during surgeryusing a pre-acquired fluoroscopic image. This is useful, as it is oftendesirable to position or implant a surgical instrument at a certainangle relative to the plane of the fluoroscopic image. For example, thesurgical instrument may need to be implanted in the direction alignedwith the medial axis of the pedicle 1503.

[0109] Consider the vertebral cross section shown as an axial CT imagein FIG. 15C. As described above, the angle 1504 between theanterior/posterior axis of the spine 1502 and the medial axis 1503 ofthe pedicle can be measured from this CT image. Aligning the surgicalinstrument with the medial axis can be accomplished by dynamicallymeasuring the angle between the trajectory of the surgical instrumentand the plane defined by the mid-line of the spine 1502. When thedynamically measured angle matches the angle pre-obtained from the CTimage, the surgical instrument is aligned.

[0110]FIGS. 16A and 16B are figures respectively illustrating ananterior/posterior fluoroscopic image of the spine and a correspondingthree-dimensional view of the spine. The physician defines two pointsalong the midline of the spine, such as the points 1601 drawn on thespinous processes in FIG. 16A (in non-pathological anatomy a spinousprocess typically defines the midline). Computer 120 uses these pointsto define a line 1602 in the image, or more generally, the computerdefines plane 1603 (shown in FIG. 16B) to include the two points and thelinear projections of these two points dictated by the calibrationtransformation. More intuitively, a first order approximation of plane1603 can be thought of as the plane passing through the two pointsperpendicular to the image plane.

[0111] Plane 1603 defines the midline of the spine in three-dimensionalspace. During navigational guidance, the equation of this plane can beexpressed in the coordinate system of either the dynamic reference frame150 or the tracking sensor 130.

[0112] Using the tracking sensor 130 to measure the position andorientation (i.e., the trajectory) of the instrument 140, computer 120then mathematically projects this trajectory onto the plane 1603. Thisprojection will define a line passing through plane 1603. The anglebetween this line in plane 1603 and the instrument trajectorycorresponds to the angle to be measured. In other words, the angle to bemeasured corresponds to the minimum angle present between the trajectoryof the instrument and the plane 1603. The angle to be measured can becalculated by computer 120 and displayed to the physician either in atextual or graphical format.

[0113] In summary, as described in this example, a single fluoroscopicimage can be used during surgery to position a surgical instrument at adesired trajectory relative to the plane of the fluoroscopic image. Moregenerally, the methods described in this example relate to measuring theangle between the trajectory of a surgical instrument 140 and a plane(e.g. 1603) defined by two or more points (e.g., 1601) which have beenmanually or automatically selected in a fluoroscopic image. While theexplanation uses a CT for clarity of the example, the measurement anddisplay of the angle can be achieved without the use of any 3D imagedata.

[0114] Although the above five examples used three-dimensional patientspecific data and not atlas data, in certain situations, it may bepossible to use a 2D/3D registration scheme that registers non-patientspecific atlas data to patient specific fluoroscopic images usingdeformable registration methods that do not preserve the rigidity ofanatomical structure during the registration process. In this manner,the patient specific fluoroscopic images may be used to deform the atlasdata to better correspond to the patient and thereby transfer atlasedknowledge to the patient specific fluoroscopic images.

Conclusion

[0115] The above described systems and methods significantly extend theconventional techniques for acquiring and using x-ray images forsurgical navigational guidance. It will be apparent to those skilled inthe art that various modifications and variations can be made to thepresent invention without departing from the scope or spirit of theinvention. For example, although certain of the examples were describedin relation to spinal examples, many other regions of body could beoperated on.

[0116] Other embodiments of the invention will be apparent to thoseskilled in the art from consideration of the specification and practiceof the invention disclosed herein. In particular, an alternativeembodiment of the calibration and tracking target may allow thecalibration component to be detached from the C-arm and introduced intothe C-arm view for calibrations only, and then removed. It is intendedthat the specification and examples be considered as exemplary only,with the true scope and spirit of the invention being indicated by thefollowing claims.

What is claimed:
 1. An x-ray imaging device comprising: an x-ray sourcefor generating cycles of x-ray radiation corresponding to an imageacquisition cycle; an x-ray receiving section positioned so that x-raysemanating from the x-ray source enter the x-ray receiving section, thex-ray receiving section generating an image representing intensities ofthe x-rays entering the x-ray receiving section; a computer coupled tothe x-ray receiving section; and radiation sensors located in a path ofx-rays emitted from the x-ray source, the radiation sensors detectingthe beginning and end of a radiation cycle and transmitting the detectedbeginning and end of the radiation cycle to the computer.
 2. The deviceof claim 1, wherein the computer further comprises a display fordisplaying the images generated by the x-ray receiving section.
 3. Thedevice of claim 1, further comprising: a dynamic reference frame markerattached to anatomy of a patient; a calibration and tracking target; anda tracking sensor for detecting the position, in three-dimensionalspace, of the dynamic reference frame marker and the calibration andtracking target; wherein the computer determines that an imageacquisition cycle is erroneous when the position of the calibration andtracking target moves relative to the position of the dynamic referenceframe during an image acquisition cycle.
 4. The device of claim 3,wherein the calibration and tracking target includes calibration markersand the calibration and tracking target is attached to the imagingdevice.
 5. The device of claim 3, wherein the calibration and trackingtarget is physically separated from the x-ray source and the x-rayreceiving section and calibration markers included in the calibrationand tracking target are in the path of x-rays of the generated cycles ofx-ray radiation.
 6. The device of claim 1, further comprisingcalibration markers for determining a projective transformation of theimage.
 7. The device of claim 6, wherein the calibration markers areused to correct image distortion.
 8. An x-ray imaging device comprising:a rotatable C-arm support having first and second ends, an x-ray sourcepositioned at the first end for initiating an imaging cycle bygenerating x-ray radiation; an x-ray receiving section positioned at thesecond end so that x-rays emanating from the x-ray source enter thex-ray receiving section, the x-ray receiving section generating an imagerepresenting the intensities of the x-rays entering the x-ray receivingsection; a calibration and tracking target; a tracking sensor fordetecting the position, in three-dimensional space, of the calibrationand tracking target; and a computer communicating with the x-rayreceiving section and the tracking sensor, the computer detecting motionof the C-arm based on changes in the position detected by the trackingsensor.
 9. The imaging device of claim 8, further comprising a dynamicreference frame marker attached to anatomy of a patient, the trackingsensor detecting the position, in three-dimensional space, of thedynamic reference frame marker.
 10. The imaging device of claim 9,further comprising means for detecting the beginning and end of animaging cycle and transmitting indications of the detected beginning andend of the radiation cycle to the computer, and the computer determiningthat an image acquisition cycle is erroneous when the position of thetracking target moves with respect to the patient during the imagingcycle.
 11. A surgical instrument navigation system comprising: acomputer processor; a tracking sensor for sensing three-dimensionalposition information of a surgical instrument and transmitting theposition information to the computer processor; a memory coupled to thecomputer processor, the memory including computer instructions that whenexecuted by the computer processor cause the processor to generate anicon representing the surgical instrument and to overlay the icon on apre-acquired x-ray image, the icon of the surgical instrumentrepresenting the real-time position of the surgical instrument projectedinto the pre-acquired x-ray image and the icon being generated as afirst representation when the surgical instrument is positioned suchthat it is substantially viewable in the plane of the pre-acquired imageand the icon being generated as a second representation when thesurgical instrument is positioned such that the surgical instrument issubstantially perpendicular to the plane of the pre-acquired image; anda display coupled to the processor for displaying the generated iconsuperimposed on the pre-acquired image.
 12. The computer system of claim11, wherein the icon is superimposed on multiple pre-acquired images.13. The computer system of claim 13, wherein the second representationincludes first and second cross-hair icons, the center of the first iconrepresenting one end of the surgical instrument and the center of thesecond icon representing an opposite end of the surgical instrument. 14.A surgical instrument navigation system comprising: a computerprocessor; a tracking sensor for sensing three-dimensional positioninformation of a surgical instrument and transmitting the positioninformation to the computer processor; a memory coupled to the computerprocessor, the memory including computer instructions that when executedby the computer processor cause the processor to generate an iconrepresenting the surgical instrument positioned in a pre-acquired imageof a patient's anatomy, the icon of the surgical instrument including afirst portion corresponding to an actual position of the surgicalinstrument and a second portion corresponding to a projection of thesurgical instrument along a line given by a current trajectory of thesurgical instrument; and a display coupled to the processor fordisplaying the generated icon superimposed on the pre-acquired image.15. The system of claim 14, wherein a length of the projection of thesurgical instrument changes based on input received from a physician.16. The computer system of claim 14, wherein the icon is superimposed onmultiple pre-acquired images.
 17. A surgical instrument navigationsystem comprising: a rotatable C-arm including an x-ray source and anx-ray receiving section for acquiring x-ray images of a patient, theC-arm being rotatable about one of a plurality of mechanical axes; acomputer processor coupled to the rotatable C-arm; a memory coupled tothe computer processor, the memory storing the x-ray images acquired bythe rotatable C-arm and computer instructions that when executed by thecomputer processor cause the computer processor to generate a linerepresenting a projection of a plane substantially parallel to one ofthe plurality of the mechanical axes of the C-arm into the x-ray image,the line enabling visual alignment of the one of the plurality ofmechanical axes of the C-arm with an axis relating complimentary x-rayimages; and a display coupled to the processor for displaying at least aportion of the generated line superimposed on the x-ray image.
 18. Thesystem of claim 17, further comprising: a tracking target attached tothe rotatable C-arm; and a tracking sensor for detecting the position,in three-dimensional space, of the tracking target, wherein the computercalculates an amount of rotation of the C-arm based on changes in theposition detected by the tracking sensor.
 19. A system for defining asurgical plan comprising: an x-ray imaging device; a surgicalinstrument; a tracking sensor for detecting the position, inthree-dimensional space, of the surgical instrument; a computerprocessor in communication with the tracking sensor for defining a pointin a virtual x-ray imaging path as the three-dimensional location of thesurgical instrument, the point being outside of an x-ray imaging path ofthe x-ray imaging device, the computer processor translating position ofthe surgical instrument within the virtual x-ray imaging path tocorresponding position in the true x-ray imaging path; and a displaycoupled to the processor for displaying a pre-acquired x-ray imageoverlaid with an iconic representation of the surgical instrument, thereal-time position of the iconic representation of the surgicalinstrument in the pre-acquired x-ray image corresponding to thetranslated position of the surgical instrument.
 20. The system of claim19, further comprising means for freezing the iconic representation ofthe surgical instrument in the x-ray image.
 21. A system for defining asurgical plan comprising: an x-ray imaging device; a surgicalinstrument; a tracking sensor for detecting the position, inthree-dimensional space, of the surgical instrument; a computerprocessor in communication with the tracking sensor for calculating aprojection of the trajectory of the surgical instrument a distance aheadof the actual location of the surgical instrument; and a display coupledto the processor for displaying a pre-acquired x-ray image overlaid withan iconic representation of the surgical instrument and the calculatedprojection of the trajectory of the surgical instrument.
 22. A systemfor realigning a first bone segment with a second bone segment in apatient comprising: a first tracking marker attached to the first bonesegment; a second tracking marker attached to the second bone segment; atracking sensor for detecting the relative position, inthree-dimensional space, of the first and second tracking markers; acomputer for delineating boundaries of images of the first and secondbone segment in a pre-acquired x-ray image and for, when the second bonesegment is moved in the patient, correspondingly moving the delineatedboundary of the second bone segment in the x-ray image; and a displaycoupled to the computer for displaying the pre-acquired x-ray imageoverlaid with representations of the delineated boundaries of the firstand second bone segments.
 23. The system of claim 22, wherein thecomputer performs edge detection processing on the x-ray image todelineate the boundaries.
 24. A system for placing a surgical implantinto a patient comprising: a computer processor; means for enteringdimensions of the implant; a tracking sensor for sensingthree-dimensional position information of a surgical instrument on whichthe surgical implant is mounted, the tracking sensor transmitting theposition information to the computer processor; and a memory coupled tothe computer processor, the memory including computer instructions thatwhen executed by the computer processor cause the processor to generatean icon representing the surgical instrument and the mounted surgicalimplant, and to overlay the icon on a pre-acquired two-dimensional x-rayimage, the icon of the surgical instrument representing the real-timeposition of the surgical instrument relative to the pre-acquiredtwo-dimensional x-ray image.
 25. The system of claim 24, wherein themeans for entering dimensions includes a display communicating with theprocessor for displaying a three-dimensional image of anatomy of thepatient through which a physician may select the appropriate size andplacement for the implant.
 26. The system of claim 24, wherein thecomputer includes means for generating a two-dimensional digitallyreconstructed radiograph (DRR) image from the three-dimensional image ofthe anatomy of the patient, the digital reconstructed radiograph imageincluding a two-dimensional representation of the appropriate size ofthe implant.
 27. The system of claim 25, wherein the surgical implant isan inter-vertebral cage.
 28. A method of acquiring a two-dimensionalx-ray image of patient anatomy from a desired view direction comprisingthe steps of: acquiring a two-dimensional image using an x-ray imager;specifying a view direction in a three-dimensional image representingthe patient anatomy; generating a two-dimensional digitallyreconstructed radiograph (DRR) image based on the three-dimensionalimage and the specified view direction; and determining whether thetwo-dimensional x-ray image corresponds to the desired view direction bymatching the DRR image to the x-ray image.
 29. The method of claim 28,wherein the steps of acquiring a two-dimensional image and determiningthat the two-dimensional x-ray image corresponds to the desired viewdirection are repeated until the two-dimensional x-ray image correspondsto the desired view direction.
 30. A method of defining a surgical plancomprising: sensing three-dimensional position information of a surgicalinstrument; generating a graphical icon representing the surgicalinstrument positioned in a pre-acquired image of a patient's anatomy,the icon of the surgical instrument including a first portioncorresponding to a position of the surgical instrument in space and asecond portion corresponding to a projection of the surgical instrumentalong a line given by a current trajectory of the surgical instrument;and displaying the generated icon superimposed on the pre-acquiredimage.
 31. The method of claim 30, further comprising freezing at leastthe first or second portion of the iconic representation of the surgicalinstrument in the x-ray image.
 32. The method of claim 30, wherein theicon is superimposed on multiple pre-acquired images.
 33. A method ofrepresenting a real-time position of a surgical instrument in apreacquired x-ray image comprising: generating an icon of a surgicalinstrument and overlaying the icon on the pre-acquired x-ray image, theicon of the surgical instrument representing the real-time position ofthe surgical instrument projected into the pre-acquired x-ray image;representing the icon as a first representation when the surgicalinstrument is positioned such that it is substantially viewable in aplane of the pre-acquired image; and representing the icon as a secondrepresentation when the surgical instrument is positioned such that itis substantially perpendicular to the plane of the pre-acquired image.34. A method of defining a surgical plan comprising: detecting aposition, in three-dimensional space, of a surgical instrument; defininga point in a virtual x-ray imaging path as the three-dimensionallocation of the surgical instrument, the point being outside of anon-virtual x-ray imaging path of the x-ray imaging device; translatingposition of the surgical instrument within the virtual x-ray imagingpath to a corresponding position in the non-virtual x-ray imaging path;and displaying a pre-acquired x-ray image overlaid with an iconicrepresentation of the surgical instrument, the position of the iconicrepresentation of the surgical instrument in the pre-acquired x-rayimage corresponding to the translated position of the surgicalinstrument.
 35. The method of claim 34, further comprising freezing atleast the first or second portion of the iconic representation of thesurgical instrument in the x-ray image.
 36. A method for realigning afirst broken bone segment with a second broken bone segment in a patientcomprising: attaching a first tracking marker to the first bone segment;attaching a second tracking marker to the second bone segment;delineating boundaries of images of the first and second bone segmentsin a pre-acquired x-ray image; detecting the relative position of thefirst and second bone segments in the patient using the first and secondtracking markers and correspondingly moving the delineated boundary ofthe second bone segment in the pre-acquired x-ray image when the secondbone segment moves relative to the first bone segment; and displayingthe pre-acquired x-ray image overlaid with representations of thedelineated boundaries of the first and second bone segments.
 37. Themethod of claim 36, further comprising the step of performing edgedetection processing on the x-ray image to delineate the boundaries. 38.A method for placing a surgical implant into a patient comprising:displaying a three-dimensional image of anatomy of the patient throughwhich an appropriately dimensioned implant is selected, the implantbeing mounted to a surgical instrument; sensing three-dimensionalposition information of the surgical instrument on which the surgicalimplant is mounted; and generating an icon representing the surgicalinstrument and the mounted surgical implant and superimposing the iconon a pre-acquired two-dimensional x-ray image, the icon of the surgicalinstrument representing the real-time position of the surgicalinstrument projected into the pre-acquired two-dimensional x-ray imageand the icon corresponding to the appropriately sized implant.
 39. Thesystem of claim 38, wherein the computer includes means for generating atwo-dimensional digitally reconstructed radiograph (DRR) image from thethree-dimensional image of the anatomy of the patient, the digitallyreconstructed radiograph including a two-dimensional representation ofthe appropriate size and placement of the implant.
 40. A method ofcalculating an angle between a surgical instrument and a plane selectedin an x-ray image, the method comprising the steps of: defining at leasttwo points in the x-ray image; defining a plane passing through thex-ray image as the plane including the two points and linear projectionsof the two points as dictated by a calibration transformation used tocalibrate the x-ray image for the particular imaging device of the x-rayimage; sensing a position of the surgical instrument inthree-dimensional space; and calculating the minimum angle between thesurgical instrument and the defined plane.
 41. The method of claim 40,wherein the plane is aligned with the mid-line of the spine of apatient.
 42. The method of claim 40, wherein the surgical instrument isrepositioned until the calculated angle equals a target angle.
 43. Themethod of claim 42, wherein the target angle is pre-determined using athree-dimensional image.
 44. A method of detecting an error in an x-rayprocess comprising: generating cycles of x-ray radiation correspondingto an image acquisition cycle, the cycles being generated by an x-rayimager and passing through calibration markers of a calibration andtracking target; generating an image of a patient's anatomy defined byintensities of the x-rays in the cycle of the x-ray radiation; detectingthe beginning and end of a radiation cycle; detecting the position ofthe calibration and tracking target and the patient; and determiningthat the image acquisition cycle is erroneous when the position of thetracking target relative to the position of the patient moves betweenthe beginning and the end of the radiation cycle.
 45. A method foraligning a fluoroscopic imager with a view direction of the medial axisof a patient's pedicle, the method comprising: displaying athree-dimensional image of an axial cross-section of vertebra of thepatient; extracting an angle from the three-dimensional imagecorresponding to the angle separating an anterior/posterior axis and themedial axis of the pedicle; aligning the fluoroscopic imager with a longaxis of the patient; and rotating the fluoroscopic imager about the longaxis of the patient through the measured angle.
 46. The method of claim45, wherein the three-dimensional image is a CT image.
 47. The method ofclaim 45, wherein rotating the fluoroscopic imager through the measuredangle includes receiving information relating to the amount of rotationof the fluoroscopic imager from a tracking sensor detecting position ofthe fluoroscopic imager.
 48. A system of defining a surgical plancomprising: means for sensing three-dimensional position information ofa surgical instrument; means for generating a graphical iconrepresenting the surgical instrument positioned in a pre-acquired imageof a patient's anatomy, the icon of the surgical instrument including afirst portion corresponding to a position of the surgical instrument inspace and a second portion corresponding to a projection of the surgicalinstrument along a line given by a current trajectory of the surgicalinstrument; and means for displaying the generated icon superimposed onthe pre-acquired image.
 49. A system for representing a real-timeposition of a surgical instrument in a pre-acquired x-ray imagecomprising: means for generating an icon of a surgical instrument andoverlaying the icon on the pre-acquired x-ray image, the icon of thesurgical instrument representing the real-time position of the surgicalinstrument projected into the pre-acquired x-ray image; means forrepresenting the icon as a first representation when the surgicalinstrument is positioned such that it is substantially viewable in aplane of the pre-acquired image; and means for representing the icon asa second representation when the surgical instrument is positioned suchthat it is substantially perpendicular to the plane of the pre-acquiredimage.
 50. A system for defining a surgical plan comprising: means fordetecting a position, in three-dimensional space, of a surgicalinstrument; means for defining a point in a virtual x-ray imaging pathas the three-dimensional location of the surgical instrument, the pointbeing outside of a non-virtual x-ray imaging path of the x-ray imagingdevice; means for translating position of the surgical instrument withinthe virtual x-ray imaging path to a corresponding position in thenon-virtual x-ray imaging path; and means for displaying a pre-acquiredx-ray image overlaid with a real-time iconic representation of thesurgical instrument, the position of the iconic representation of thesurgical instrument in the pre-acquired x-ray image corresponding to thetranslated position of the surgical instrument.
 51. A system forrealigning a first broken bone segment with a second broken bone segmentin a patient comprising: means for attaching a first tracking marker tothe first bone segment; means for attaching a second tracking marker tothe second bone segment; means for delineating boundaries of images ofthe first and second bone segments in a pre-acquired x-ray image; meansfor detecting the relative position of the first and second bonesegments in the patient using the first and second tracking markers andcorrespondingly moving the delineated boundary of the second bonesegment in the pre-acquired x-ray image when the second bone segmentmoves relative to the first bone segment; and means for displaying thepre-acquired x-ray image overlaid with representations of the delineatedboundaries of the first and second bone segments.
 52. A system forcalculating an angle between a surgical instrument and a plane selectedin an x-ray image, the system comprising: means for defining at leasttwo points in the x-ray image; means for defining a plane passingthrough the x-ray image as the plane including the two points and linearprojections of the two points as dictated by a calibrationtransformation used to calibrate the x-ray image for the particularimaging device of the x-ray image; means for sensing a position of thesurgical instrument in three-dimensional space; and means forcalculating the angle between the surgical instrument and the definedplane.
 53. An x-ray imaging device comprising: means for generatingcycles of x-ray radiation corresponding to an image acquisition cycle;means for generating an image representing intensities of the x-raysentering the x-ray receiving section; and means for detecting thebeginning and end of a radiation cycle and transmitting the detectedbeginning and end of the radiation cycle to the computer.